Improving “Fast Iterative Shrinkage-Thresholding Algorithm”: Faster, Smarter, and Greedier

The “fast iterative shrinkage-thresholding algorithm,” a.k.a. FISTA, is one of the most well known first-order optimization scheme in literature, as it achieves worst-case $O(1/k^2)$ optimal convergence rate for objective function value. However, despite such an theoretical rate, practice (local) oscillatory behavior FISTA often damps its efficiency. Over past years, various efforts have been made literature to improve practical performance monotone restarting and backtracking strategies. In this paper, we propose a simple yet effective modification original which has two advantages: It allows us (1) prove generated sequence (2) design so-called lazy-start strategy can be up order faster than scheme. Moreover, novel adaptive greedy strategies further performance. advantages proposed schemes are tested through problems arising from inverse problems, machine learning, signal/image processing.